Generalized and transferable patient language representation for phenotyping with limited data

The paradigm of representation learning through transfer has the potential to greatly enhance clinical natural language processing. In this work, we propose a multi-task pre-training and fine-tuning approach for generalized transferable patient representations from medical language. model is first pre-trained with different but related high-prevalence phenotypes further fine-tuned on downstream target tasks. Our main contribution focuses impact technique can have low-prevalence phenotypes, challenging task due dearth data. We validate pre-training, fine-tune models including 38 circulatory diseases, 23 respiratory 17 genitourinary diseases. find increases efficiency achieves consistently high performance across majority phenotypes. Most important, almost always either best-performing or performs tolerably close model, property refer as robust. All these results lead us conclude that architecture robust developing numerous